We indicate in path/pathq/url that they may contain '#' if the frontend
is configured with "option accept-invalid-http-request", and that option
mentions the fragment as well.
Since we now block fragments by default, let's add an extra test there
to confirm that it's blocked even when stripping it.
(cherry picked from commit 4d0175b54b2b4eeb01aa6e31282b0a5b0d7d8ace)
[ad: backported to test conformance of BUG/MINOR: h1: do not accept '#'
as part of the URI component] Signed-off-by: Amaury Denoyelle <adenoyelle@haproxy.com>
(cherry picked from commit b3f26043df74c661155566a0abd56103e8116078) Signed-off-by: Amaury Denoyelle <adenoyelle@haproxy.com>
(cherry picked from commit 41d161ccbbfa846b4b17ed0166ff08f6bf0c3ea1) Signed-off-by: Amaury Denoyelle <adenoyelle@haproxy.com>
(cherry picked from commit b6b330eb117d520a890e5b3cd623eaa73479db1b) Signed-off-by: Willy Tarreau <w@1wt.eu>
(cherry picked from commit 73b9b13ac2654ef5384789685e3d65ca5f2f880a)
[wt: rewrote the test for 2.2 without normalize-uri and called it
fragments-in-uri] Signed-off-by: Willy Tarreau <w@1wt.eu>
Gbp-Pq: Name REGTESTS-http-rules-verify-that-we-block-by-default-.patch
BUG/MINOR: h1: do not accept '#' as part of the URI component
In addition to the current NUL/CR/LF, this will also reject all other
control chars, the space and '#' from the :path pseudo-header, to avoid
taking the '#' for a part of the path. It's still possible to fall back
to the previous behavior using "option accept-invalid-http-request".
This patch modifies the request parser to change the ":path" pseudo header
validation function with a new one that rejects 0x00-0x1F (control chars),
space and '#'. This way such chars will be dropped early in the chain, and
the search for '#' doesn't incur a second pass over the header's value.
This should be progressively backported to stable versions, along with the
following commits it relies on:
REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests
REORG: http: move has_forbidden_char() from h2.c to http.h
MINOR: ist: add new function ist_find_range() to find a character range
MINOR: http: add new function http_path_has_forbidden_char()
MINOR: h2: pass accept-invalid-http-request down the request parser
Seth Manesse and Paul Plasil reported that the "path" sample fetch
function incorrectly accepts '#' as part of the path component. This
can in some cases lead to misrouted requests for rules that would apply
on the suffix:
Note that this behavior can be selectively configured using
"normalize-uri fragment-encode" and "normalize-uri fragment-strip".
The problem is that while the RFC says that this '#' must never be
emitted, as often it doesn't suggest how servers should handle it. A
diminishing number of servers still do accept it and trim it silently,
while others are rejecting it, as indicated in the conversation below
with other implementers:
Looking at logs from publicly exposed servers, such requests appear at
a rate of roughly 1 per million and only come from attacks or poorly
written web crawlers incorrectly following links found on various pages.
Thus it looks like the best solution to this problem is to simply reject
such ambiguous requests by default, and include this in the list of
controls that can be disabled using "option accept-invalid-http-request".
We're already rejecting URIs containing any control char anyway, so we
should also reject '#'.
In the H1 parser for the H1_MSG_RQURI state, there is an accelerated
parser for bytes 0x21..0x7e that has been tightened to 0x24..0x7e (it
should not impact perf since 0x21..0x23 are not supposed to appear in
a URI anyway). This way '#' falls through the fine-grained filter and
we can add the special case for it also conditionned by a check on the
proxy's option "accept-invalid-http-request", with no overhead for the
vast majority of valid URIs. Here this information is available through
h1m->err_pos that's set to -2 when the option is here (so we don't need
to change the API to expose the proxy). Example with a trivial GET
through netcat:
[08/Aug/2023:16:16:52.651] frontend layer1 (#2): invalid request
backend <NONE> (#-1), server <NONE> (#-1), event #0, src 127.0.0.1:50812
buffer starts at 0 (including 0 out), 16361 free,
len 23, wraps at 16336, error at position 7
H1 connection flags 0x00000000, H1 stream flags 0x00000810
H1 msg state MSG_RQURI(4), H1 msg flags 0x00001400
H1 chunk len 0 bytes, H1 body len 0 bytes :
00000 GET /aa#bb HTTP/1.0\r\n
00021 \r\n
This should be progressively backported to all stable versions along with
the following patch:
REGTESTS: http-rules: add accept-invalid-http-request for normalize-uri tests
Similar fixes for h2 and h3 will come in followup patches.
Thanks to Seth Manesse and Paul Plasil for reporting this problem with
detailed explanations.
We're adding a new argument "relaxed" to h2_make_htx_request() so that
we can control its level of acceptance of certain invalid requests at
the proxy level with "option accept-invalid-http-request". The goal
will be to add deactivable checks that are still desirable to have by
default. For now no test is subject to it.
As its name implies, this function checks if a path component has any
forbidden headers starting at the designated location. The goal is to
seek from the result of a successful ist_find_range() for more precise
chars. Here we're focusing on 0x00-0x1F, 0x20 and 0x23 to make sure
we're not too strict at this point.
(cherry picked from commit 30f58f4217d585efeac3d85cb1b695ba53b7760b)
[ad: backported for following fix : BUG/MINOR: h2: reject more chars
from the :path pseudo header] Signed-off-by: Amaury Denoyelle <adenoyelle@haproxy.com>
(cherry picked from commit b491940181a88bb6c69ab2afc24b93a50adfa67c) Signed-off-by: Amaury Denoyelle <adenoyelle@haproxy.com>
(cherry picked from commit f7666e5e43ce63e804ebffdf224d92cfd3367282) Signed-off-by: Amaury Denoyelle <adenoyelle@haproxy.com>
(cherry picked from commit c699bb17b7e334c9d56e829422e29e5a204615ec)
[wt: adj minor ctx in http.h] Signed-off-by: Willy Tarreau <w@1wt.eu>
(cherry picked from commit 0f57ac20b046b70275192651d7b6c978032e6a36)
[wt: adj minor ctx in http.h] Signed-off-by: Willy Tarreau <w@1wt.eu>
Gbp-Pq: Name MINOR-http-add-new-function-http_path_has_forbidden_.patch
(cherry picked from commit cf26623780bdd66f4fff4154d0e5081082aff89b)
[wt: needed for next fix] Signed-off-by: Willy Tarreau <w@1wt.eu>
Gbp-Pq: Name MINOR-ist-Add-istend-function-to-return-a-pointer-to.patch
This looks up the character range <min>..<max> in the input string and
returns a pointer to the first one found. It's essentially the equivalent
of ist_find_ctl() in that it searches by 32 or 64 bits at once, but deals
with a range.
The content-length header parser has its dedicated function, in order
to take extreme care about invalid, unparsable, or conflicting values.
But there's a corner case in it, by which it stops comparing values
when reaching the end of the header. This has for a side effect that
an empty value or a value that ends with a comma does not deserve
further analysis, and it acts as if the header was absent.
While this is not necessarily a problem for the value ending with a
comma as it will be cause a header folding and will disappear, it is a
problem for the first isolated empty header because this one will not
be recontructed when next ones are seen, and will be passed as-is to the
backend server. A vulnerable HTTP/1 server hosted behind haproxy that
would just use this first value as "0" and ignore the valid one would
then not be protected by haproxy and could be attacked this way, taking
the payload for an extra request.
In field the risk depends on the server. Most commonly used servers
already have safe content-length parsers, but users relying on haproxy
to protect a known-vulnerable server might be at risk (and the risk of
a bug even in a reputable server should never be dismissed).
A configuration-based work-around consists in adding the following rule
in the frontend, to explicitly reject requests featuring an empty
content-length header that would have not be folded into an existing
one:
http-request deny if { hdr_len(content-length) 0 }
The real fix consists in adjusting the parser so that it always expects a
value at the beginning of the header or after a comma. It will now reject
requests and responses having empty values anywhere in the C-L header.
This needs to be backported to all supported versions. Note that the
modification was made to functions h1_parse_cont_len_header() and
http_parse_cont_len_header(). Prior to 2.8 the latter was in
h2_parse_cont_len_header(). One day the two should be refused but the
former is also used by Lua.
The HTTP messaging reg-tests were completed to test these cases.
Thanks to Ben Kallus of Dartmouth College and Narf Industries for
reporting this! (this is in GH #2237).
Willy Tarreau [Thu, 9 Feb 2023 20:36:54 +0000 (21:36 +0100)]
BUG/CRITICAL: http: properly reject empty http header field names
The HTTP header parsers surprizingly accepts empty header field names,
and this is a leftover from the original code that was agnostic to this.
When muxes were introduced, for H2 first, the HPACK decompressor needed
to feed headers lists, and since empty header names were strictly
forbidden by the protocol, the lists of headers were purposely designed
to be terminated by an empty header field name (a principle that is
similar to H1's empty line termination). This principle was preserved
and generalized to other protocols migrated to muxes (H1/FCGI/H3 etc)
without anyone ever noticing that the H1 parser was still able to deliver
empty header field names to this list. In addition to this it turns out
that the HPACK decompressor, despite a comment in the code, may
successfully decompress an empty header field name, and this mistake
was propagated to the QPACK decompressor as well.
The impact is that an empty header field name may be used to truncate
the list of headers and thus make some headers disappear. While for
H2/H3 the impact is limited as haproxy sees a request with missing
headers, and headers are not used to delimit messages, in the case of
HTTP/1, the impact is significant because the presence (and sometimes
contents) of certain sensitive headers is detected during the parsing.
Thus, some of these headers may be seen, marked as present, their value
extracted, but never delivered to upper layers and obviously not
forwarded to the other side either. This can have for consequence that
certain important header fields such as Connection, Upgrade, Host,
Content-length, Transfer-Encoding etc are possibly seen as different
between what haproxy uses to parse/forward/route and what is observed
in http-request rules and of course, forwarded. One direct consequence
is that it is possible to exploit this property in HTTP/1 to make
affected versions of haproxy forward more data than is advertised on
the other side, and bypass some access controls or routing rules by
crafting extraneous requests. Note, however, that responses to such
requests will normally not be passed back to the client, but this can
still cause some harm.
This specific risk can be mostly worked around in configuration using
the following rule that will rely on the bug's impact to precisely
detect the inconsistency between the known body size and the one
expected to be advertised to the server (the rule works from 2.0 to
2.8-dev):
http-request deny if { fc_http_major 1 } !{ req.body_size 0 } !{ req.hdr(content-length) -m found } !{ req.hdr(transfer-encoding) -m found } !{ method CONNECT }
This will exclusively block such carefully crafted requests delivered
over HTTP/1. HTTP/2 and HTTP/3 do not need content-length, and a body
that arrives without being announced with a content-length will be
forwarded using transfer-encoding, hence will not cause discrepancies.
In HAProxy 2.0 in legacy mode ("no option http-use-htx"), this rule will
simply have no effect but will not cause trouble either.
A clean solution would consist in modifying the loops iterating over
these headers lists to check the header name's pointer instead of its
length (since both are zero at the end of the list), but this requires
to touch tens of places and it's very easy to miss one. Functions such
as htx_add_header(), htx_add_trailer(), htx_add_all_headers() would be
good starting points for such a possible future change.
Instead the current fix focuses on blocking empty headers where they
are first inserted, hence in the H1/HPACK/QPACK decoders. One benefit
of the current solution (for H1) is that it allows "show errors" to
report a precise diagnostic when facing such invalid HTTP/1 requests,
with the exact location of the problem and the originating address:
I want to address sincere and warm thanks for their great work to the
team composed of the following security researchers who found the issue
together and reported it: Bahruz Jabiyev, Anthony Gavazzi, and Engin
Kirda from Northeastern University, Kaan Onarlioglu from Akamai
Technologies, Adi Peleg and Harvey Tuch from Google. And kudos to Amaury
Denoyelle from HAProxy Technologies for spotting that the HPACK and
QPACK decoders would let this pass despite the comment explicitly
saying otherwise.
This fix must be backported as far as 2.0. The QPACK changes can be
dropped before 2.6. In 2.0 there is also the equivalent code for legacy
mode, which doesn't suffer from the list truncation, but it would better
be fixed regardless.
Gbp-Pq: Name 2.0-2.5-BUG-CRITICAL-http-properly-reject-empty-http-header-.patch
As state in RFC9113#8.1, HEADERS frame with the ES flag set that carries an
informational status code is malformed. However, there is no test on this
condition.
On 2.4 and higher, it is hard to predict consequences of this bug because
end of the message is only reported with a flag. But on 2.2 and lower, it
leads to a crash because there is an unexpected extra EOM block at the end
of an interim response.
Now, when a ES flag is detected on a HEADERS frame for an interim message, a
stream error is sent (RST_STREAM/PROTOCOL_ERROR).
This patch should solve the issue #1972. It should be backported as far as
2.0.
Ensure calls to http_find_header() terminate. If a "Set-Cookie2"
header is found then the while(1) loop in
http_manage_server_side_cookies() will never terminate, resulting in
the watchdog firing and the process terminating via SIGABRT.
The while(1) loop becomes unbounded because an unmatched call to
http_find_header("Set-Cookie") will leave ctx->blk=NULL. Subsequent
calls to check for "Set-Cookie2" will now enumerate from the beginning
of all the blocks and will once again match on subsequent
passes (assuming a match first time around), hence the loop becoming
unbounded.
This issue was introduced with HTX and this fix should be backported
to all versions supporting HTX.
Many thanks to Grant Spence (gspence@redhat.com) for working through
this issue with me.
Willy Tarreau [Thu, 26 Aug 2021 14:23:37 +0000 (16:23 +0200)]
BUG/MAJOR: htx: fix missing header name length check in htx_add_header/trailer
Shachar Menashe for JFrog Security reported that htx_add_header() and
htx_add_trailer() were missing a length check on the header name. While
this does not allow to overwrite any memory area, it results in bits of
the header name length to slip into the header value length and may
result in forging certain header names on the input. The sad thing here
is that a FIXME comment was present suggesting to add the required length
checks :-(
The injected headers are visible to the HTTP internals and to the config
rules, so haproxy will generally stay synchronized with the server. But
there is one exception which is the content-length header field, because
it is already deduplicated on the input, but before being indexed. As
such, injecting a content-length header after the deduplication stage
may be abused to present a different, shorter one on the other side and
help build a request smuggling attack, or even maybe a response splitting
attack.
As a mitigation measure, it is sufficient to verify that no more than
one such header is present in any message, which is normally the case
thanks to the duplicate checks:
http-request deny if { req.hdr_cnt(content-length) gt 1 }
http-response deny if { res.hdr_cnt(content-length) gt 1 }
This must be backported to all HTX-enabled versions, hence as far as 2.0.
In 2.3 and earlier, the functions are in src/htx.c instead.
Many thanks to Shachar for his work and his responsible report!
[wt: code is in src/htx.c in 2.3 and older] Signed-off-by: Willy Tarreau <w@1wt.eu>
Gbp-Pq: Name 0001-2.0-2.3-BUG-MAJOR-htx-fix-missing-header-name-length-check-i.patch
Willy Tarreau [Thu, 19 Aug 2021 21:06:58 +0000 (23:06 +0200)]
[PATCH] BUG/MEDIUM: h2: match absolute-path not path-absolute for :path
RFC7540 states that :path follows RFC3986's path-absolute. However
that was a bug introduced in the spec between draft 04 and draft 05
of the spec, which implicitly causes paths starting with "//" to be
forbidden. HTTP/1 (and now HTTP core semantics) made it explicit
that the request-target in origin-form follows a purposely defined
absolute-path defined as 1*(/ segment) to explicitly allow "//".
http2bis now fixes this by relying on absolute-path so that "//"
becomes valid and matches other versions. Full discussion here:
This issue appeared in haproxy with commit 4b8852c70 ("BUG/MAJOR: h2:
verify that :path starts with a '/' before concatenating it") when
making the checks on :path fully comply with the spec, and was backported
as far as 2.0, so this fix must be backported there as well to allow
"//" in H2 again.
Willy Tarreau [Wed, 11 Aug 2021 13:39:13 +0000 (15:39 +0200)]
BUG/MEDIUM: h2: give :authority precedence over Host
The wording regarding Host vs :authority in RFC7540 is ambiguous as it
says that an intermediary must produce a host header from :authority if
Host is missing, but doesn't say anything regarding the possibility that
Host and :authority differ, which leaves Host with higher precedence
there. In addition it mentions that clients should use :authority
*instead* of Host, and that H1->H2 should use :authority only if the
original request was in authority form. This leaves some gray area in
the middle of the chain for fully valid H2 requests arboring a Host
header that are forwarded to the other side where it's possible to
drop the Host header and use the authority only after forwarding to a
second H2 layer, thus possibly seeing two different values of Host at
a different stage. There's no such issue when forwarding from H2 to H1
as the authority is dropped only only the Host is kept.
Note that the following request is sufficient to re-normalize such a
request:
http-request set-header host %[req.hdr(host)]
The new spec in progress (draft-ietf-httpbis-http2bis-03.html) addresses
this trouble by being a bit is stricter on these rules. It clarifies that
:authority must always be used instead of Host and that Host ought to be
ignored. This is much saner as it avoids to convey two distinct values
along the chain. This becomes the protocol-level equivalent of:
http-request set-uri %[url]
So this patch does exactly this, which we were initially a bit reluctant
to do initially by lack of visibiility about other implementation's
expectations. In addition it slightly simplifies the Host header field
creation by always placing it first in the list of headers instead of
last; this could speed up the look up a little bit.
This needs to be backported to 2.0. The legacy code there is a bit
different but not too much, the code can be simplified to simply
omit host during the copy and continue.
Willy Tarreau [Wed, 11 Aug 2021 09:12:46 +0000 (11:12 +0200)]
BUG/MAJOR: h2: enforce checks on the method syntax before translating to HTX
The situation with message components in H2 is always troubling. They're
produced by the HPACK layer which contains a dictionary of well-known
hardcoded values, yet wants to remain binary transparent and protocol-
agnostic with HTTP just being one user, yet at the H2 layer we're
supposed to enforce some checks on some selected pseudo-headers that
come from internal constants... The :method pseudo-header is no exception
and is not tested when coming from the HPACK layer. This makes it possible
to pass random chars into methods, that can be serialized on another H2
connection (where they would not harm), or worse, on an H1 connection
where they can be used to transform the forwareded request. This is
similar to the request line injection described here:
https://portswigger.net/research/http2
A workaround here is to reject malformed methods by placing this rule
in the frontend or backend, at least before leaving haproxy in H1:
http-request reject if { method -m reg [^A-Z0-9] }
Alternately H2 may be globally disabled by commenting out the "alpn"
directive on "bind" lines, and by rejecting H2 streams creation by
adding the following statement to the global section:
tune.h2.max-concurrent-streams 0
This patch adds a check for each character of the method to be part of
the ones permitted in a token, as mentioned in RFC7231#4.1. This should
be backported to versions 2.0 and above, maybe even 1.8. For older
versions not having HTX_FL_PARSING_ERROR, a "goto fail" works as well
as it results in a protocol error at the stream level. Non-HTX versions
were initially thought to be safe but must be carefully rechecked since
they transcode the request into H1 before processing it.
Willy Tarreau [Tue, 10 Aug 2021 14:30:55 +0000 (16:30 +0200)]
BUG/MAJOR: h2: verify that :path starts with a '/' before concatenating it
Tim Düsterhus found that while the H2 path is checked for non-emptiness,
invalid chars and '*', a test is missing to verify that except for '*',
it always starts with exactly one '/'. During the reconstruction of the
full URI when passing to HTX, this allows to affect the apparent authority
by appending a port number or a suffix name.
This only affects H2-to-H2 communications, as H2-to-H1 do not use the
authority. Like for previous fix, the following rule installed in the
frontend or backend is sufficient to renormalize the internal URI:
http-request set-header host %[req.hdr(host)]
This needs to be backported to 2.2, since earlier versions do not rebuild
a full URI using the authority and will fail on the malformed path at the
HTTP layer.
Willy Tarreau [Tue, 10 Aug 2021 13:37:34 +0000 (15:37 +0200)]
BUG/MAJOR: h2: verify early that non-http/https schemes match the valid syntax
While we do explicitly check for strict character sets in the scheme,
this is only done when extracting URL components from an assembled one,
and we have special handling for "http" and "https" schemes directly in
the H2-to-HTX conversion. Sadly, this lets all other ones pass through
if they start exactly with "http://" or "https://", allowing the
reconstructed URI to start with a different looking authority if it was
part of the scheme.
It's interesting to note that in this case the valid authority is in
the Host hedaer and that the request will only be wrong if emitted over
H2 on the backend side, since H1 will not emit an absolute URI by
default and will drop the scheme. So in essence, this is a variant of
the scheme-based attack described below in that it only affects H2-H2
and not H2-H1 forwarding:
https://portswigger.net/research/http2
As such, a simple workaround consists in just adding the following
rule in the frontend or backend, which will have for effect to
renormalize the authority in the request line according to the
concatenated version:
http-request set-uri %[url]
This patch simply adds the missing syntax checks for non-http/https
schemes before the concatenation in the H2 code. An improvement may
consist in the future in splitting these ones apart in the start
line so that only the "url" sample fetch function requires to access
them together and that all other places continue to access them
separately. This will then allow the core code to perform such checks
itself.
The patch needs to be backported as far as 2.2. Before 2.2 the full
URI was not being reconstructed so the scheme and authority part were
always dropped from H2 requests to leave only origin requests. Note
for backporters: this depends on this previous patch:
MINOR: http: add a new function http_validate_scheme() to validate a scheme
Many thanks to Tim Düsterhus for figuring that one and providing a
reproducer.
Willy Tarreau [Tue, 10 Aug 2021 13:35:36 +0000 (15:35 +0200)]
MINOR: http: add a new function http_validate_scheme() to validate a scheme
While http_parse_scheme() extracts a scheme from a URI by extracting
exactly the valid characters and stopping on delimiters, this new
function performs the same on a fixed-size string.
[PATCH] BUG/MINOR: tcpcheck: Update .health threshold of agent inside an agent-check
If an agent-check is configured for a server, When the response is parsed,
the .health threshold of the agent must be updated on up/down/stopped/fail
command and not the threshold of the health-check. Otherwise, the
agent-check will compete with the health-check and may mark a DOWN server as
UP.
This patch should fix the issue #1176. It must be backported as far as 2.2.
(cherry picked from commit 24ec9434271345857b42cc5bd9c6b497ab01a7e4) Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
(cherry picked from commit 789bbdc88d7ffe8f520532efb18148ea52ede4ca) Signed-off-by: Christopher Faulet <cfaulet@haproxy.com>
Gbp-Pq: Name 0001-BUG-MINOR-tcpcheck-Update-.health-threshold-of-agent.patch
As HAProxy is running chrooted by default, we rely on an additional syslog
socket created by rsyslog inside the chroot for logging. As this socket cannot
trigger syslog activation, we explicitly order HAProxy after rsyslog.service.
Note that we are not using syslog.service here, since the additional socket is
rsyslog-specific.
Forwarded: no
Last-Update: 2017-12-01
Gbp-Pq: Name haproxy.service-start-after-syslog.patch
* Non-maintainer upload by the Security Team.
* BUG/MAJOR: http: reject any empty content-length header value
(CVE-2023-40225) (Closes: #1043502)
* MINOR: ist: add new function ist_find_range() to find a character range
* MINOR: ist: Add istend() function to return a pointer to the end of the
string
* MINOR: http: add new function http_path_has_forbidden_char()
* MINOR: h2: pass accept-invalid-http-request down the request parser
* BUG/MINOR: h1: do not accept '#' as part of the URI component
(CVE-2023-45539)
* BUG/MINOR: h2: reject more chars from the :path pseudo header
* REGTESTS: http-rules: verify that we block '#' by default for
normalize-uri
* DOC: clarify the handling of URL fragments in requests
projects / haproxy.git / log